Certifiable Distributional Robustness with Principled Adversarial Training

Neural networks are vulnerable to adversarial examples and researchers have proposed manyheuristic attack and defense mechanisms. We take the principled view of distributionally ro-bust optimization, which guarantees performance under adversarial input perturbations. Byconsidering a Lagrangian penalty formulation of perturbation of the underlying data distribu-tion in a Wasserstein ball, we provide a training procedure that augments model parameterupdates with worst-case perturbations of training data. For smooth losses, our procedure prov-ably achieves moderate levels of robustness with little computational or statistical cost relativeto empirical risk minimization. Furthermore, our statistical guarantees allow us to efficientlycertify robustness for the population loss. For imperceptible perturbations, our method matchesor outperforms heuristic approaches.